<!DOCTYPE html>
<html lang="zh-cn">
<head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <title>openGauss Blog  | openGauss备库wal replay与query冲突</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">

    
    <link rel="shortcut icon" href="https://xzx666.gitee.io/lookeng/img/favicon.ico" type="image/x-icon" />
    <link rel="apple-touch-icon" href="https://xzx666.gitee.io/lookeng/img/apple-touch-icon.png" />

    <link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">

    
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css">
    <script src="https://cdn.bootcss.com/jquery/3.4.1/jquery.js"></script>
    
    <link rel="stylesheet" href="https://xzx666.gitee.io/lookeng/css/commen.css?t=1607593672000" />
    <link rel="stylesheet" href="https://xzx666.gitee.io/lookeng/css/blog.css?t=1607593672000" />
    <link rel="stylesheet" href="https://xzx666.gitee.io/lookeng/css/mobile.css?t=1607593672000" />
    <link rel="stylesheet" href="https://xzx666.gitee.io/lookeng/css/pagination.css?t=1607593672000">
    
    <script>
      var _hmt = _hmt || [];
      (function() {
        var hm = document.createElement("script");
        hm.src = "https://hm.baidu.com/hm.js?34e9f585f29581007941aa1698181871";
        var s = document.getElementsByTagName("script")[0]; 
        s.parentNode.insertBefore(hm, s);
      })();
    </script>
</head>
<body>
  <script src="//cdn1.lncld.net/static/js/3.0.4/av-min.js"></script>
  <script src='//unpkg.com/valine/dist/Valine.min.js'></script>






<div class="container">
    <div class="columns blog-detail">
        <div class="post_detail">
            <div class="is-child box">
                <div class="breadCrumb"><a href="\zh\">博客/</a></div>
                <div class="blog-detail-header">
                    <h1>openGauss备库wal replay与query冲突</h1>
                    <div class="blog-detail-prop">
                        <img src="https://xzx666.gitee.io/lookeng/img/icon-date.png">
                        <span class="article_right_date">2020-11-25</span>
                        <img src="https://xzx666.gitee.io/lookeng/img/icon-author.png">
                        <span class="article_right_author">高云龙</span>
                        <img src="https://xzx666.gitee.io/lookeng/img/icon-view.png">
                        <span id="/zh/post/gaoyunlong/opengauss%E5%A4%87%E5%BA%93wal-replay%E4%B8%8Equery%E5%86%B2%E7%AA%81/" class="leancloud_visitors" data-flag-title="openGauss备库wal replay与query冲突">
                            <i class="leancloud-visitors-count"></i>                     
                        </span>
                    </div>
                    <div class="blog-detail-tags">
                        <img src="https://xzx666.gitee.io/lookeng/img/icon-tag.png">
                        
                        <a href="https://xzx666.gitee.io/lookeng/zh/tags/opengauss%E6%95%85%E9%9A%9C%E5%A4%84%E7%90%86">openGauss故障处理</a>
                        
                    </div>
                </div>
                <div class="content blog-content">
                    

<h1 id="opengauss备库wal-replay与query冲突-a-name-zh-cn-topic-0293240560-a">openGauss备库wal replay与query冲突<a name="ZH-CN_TOPIC_0293240560"></a></h1>

<h2 id="概述-a-name-section15222171311307-a">概述<a name="section15222171311307"></a></h2>

<p>openGauss的物理流复制逻辑继承了PostgreSQL，当一条数据从主库做变更到可以在备库查询到最新的值，在PostgreSQL备库分为三个阶段，分别是写入备库操作系统（remote_write），将缓存中的数据刷入到磁盘（on == flush），从磁盘将数据库回放(remote_apply)；在openGauss（自编译1116版本）备库中的三个阶段分别是接收wal（remote_receive），写入(remote_write)，回放（remote_apply）。</p>

<p>日常生产环境中，为了分担主库的访问压力，备库（hot_standby=on）常常需要对外提供只读服务，此时备库既要接受并重放通过流协议传过来的wal数据，同时也要对外提供只读查询服务，这两个任务同时进行难免会产生冲突，有时我们会发现备库的查询直接被取消中断，当数据库集群备库中出现如下所示的报错。</p>

<pre><code>ERROR: canceling statement due to conflict with recovery
Detail: User query might have needed to see row versions that must be removed
</code></pre>

<p>当日志中出现这个报错，说明wal apply的操作与query冲突，查询的sql被取消，按常规来说wal 数据回放级别应该是最高的，要保证备库第一时间获取并重放最新数据，与主库的数据保持一致，但是考虑到备库可以提供对外只读查询服务，添加参数max_standby_streaming_delay 来告诉备库，如果发现与备库当前查询冲突，需要等待max_standby_streaming_delay的时长，如果超过这个时长，查询还没结束，就直接取消掉查询操作，而这种冲突常常出现在主库执行vacuum和ddl操作。</p>

<h2 id="冲突原因-a-name-section138214453114-a">冲突原因<a name="section138214453114"></a></h2>

<p>出现冲突报错的原因可能是由以下几种情况产生：</p>

<ol>
<li>max_standby_streaming_delay参数值设置过低PostgreSQL的默认值是30s，openGauss的默认值是3s，这个值需根据不同的业务应用去设置。</li>
<li>备库sql执行时间过长由于数据量增加或数据倾斜、表或索引膨胀等因素导致sql执行变慢。</li>
<li>备机服务器压力过大，处理慢主库开启了full_page_writes，wal的数据量变大，备库需要处理的数据增多，是备库的负载增加。</li>
</ol>

<h2 id="优化建议-a-name-section146054193519-a">优化建议<a name="section146054193519"></a></h2>

<ol>
<li><p>调整参数值</p>

<ol>
<li>适当增加max_standby_streaming_delay参数值，根据使用场景自定义设置。</li>
<li>设置参数hot_standby_feedback = on，告诉主库执行vacuum操作时需要跳过哪些历史数据（不建议使用，会造成主库表膨胀）。</li>
<li>主节点设置vacuum_defer_cleanup_age参数，延迟清理dead row（不建议使用，不好控制有效值）。</li>
<li>关闭full_page_writes参数，开启enable_double_write和enable_incremental_checkpoint参数来替代，openGauss特性。</li>
</ol></li>

<li><p>优化sql可通过执行计划查找sql变慢，执行效率变差的原因，并修复。</p></li>

<li><p>提升硬件配置查看服务器硬件资源使用情况，并做适当优化及调整。</p></li>

<li><p>开启并发apply（opengauss特有）开启并发回放参数recovery_max_workers（默认值是1，最大值是4），来快速恢复wal数据并重放，提升效率。</p></li>
</ol>

                </div>
            </div>
            <div class="box" style="margin-top:15px;">
                <div style="font-size:14px;color:gray"><strong>【免责声明】</strong>本文仅代表作者本人观点，与本网站无关。本网站对文中陈述、观点判断保持中立，不对所包含内容的准确性、可靠性或完整性提供任何明示或暗示的保证。本文仅供读者参考，由此产生的所有法律责任均由读者本人承担。</div>
            </div>
            <div class="post-comment">
                
                  
  <link rel="stylesheet" href="https://xzx666.gitee.io/lookeng/css/custom.css" />
  <div id="vcomments" lang="zh-cn" mess="说点什么吧..."></div>

  <script type="text/javascript">
    const placeholder = document.getElementById("vcomments").getAttribute("mess"),
          lang = document.getElementById("vcomments").getAttribute("lang");
    const  langs = lang == "zh-cn" ? "zh-cn" : "en";
    new Valine({
        el: '#vcomments' ,
        appId: '6wfgavgIRqmpC3hjHqQVtFWF-gzGzoHsz',
        appKey: 'QRqrBDBB0p0YhrGe9IJ169ip',
        avatar:'mm', 
        placeholder: placeholder,
        visitor: true,
		    meta: ['nick','mail','link'],
        lang: langs
    });
  </script>
            </div>
        </div>
    </div>
</div>
<script>
    $(function (){
        var query = new AV.Query('Counter');
        query.equalTo('url', decodeURIComponent('\/zh\/post\/gaoyunlong\/opengauss%E5%A4%87%E5%BA%93wal-replay%E4%B8%8Equery%E5%86%B2%E7%AA%81\/'));
        query.find().then(function (data) {
            if(data.length && (data[0].attributes.author === 'openGauss')){
                var counter = AV.Object.createWithoutData('Counter', data[0].id);
                counter.set('author', '高云龙');
                counter.save();
            }
        })
    })
</script>







<input id="iframeUrl" type="text" style="display: none;" value=https://xzx666.gitee.io/>




<script defer src="https://xzx666.gitee.io/lookengjs/all.js"></script>
<script src="https://xzx666.gitee.io/lookengjs/flexible.js"></script>
<script>
  const hash = window.location.search,
        pageurl = window.location.href;
  var langss = document.querySelector("html").lang === "zh-cn" ? "zh" : "en";
  console.log("hash:",hash,"pageurl:",pageurl,"ttr",pageurl.split(langss + "/")[1])

  if(!document.getElementById("vcomments")) {
    new Valine({
        appId: '6wfgavgIRqmpC3hjHqQVtFWF-gzGzoHsz',
        appKey: 'QRqrBDBB0p0YhrGe9IJ169ip',
        avatar:'mm',
        visitor: true,
		    meta: ['nick','mail','link']
    });
  }
 
  
    


    const url = document.getElementById("iframeUrl").value;

    function observe (el, options, callback) {
      var MutationObserver = window.MutationObserver || window.WebKitMutationObserver || window.MozMutationObserver
      var observer = new MutationObserver(callback)
      observer.observe(el, options)
    }

    var options = {
        childList: true,
        subtree: true,
        characterData: true
      }
    observe(document.body, options, (records, instance) => {
    const height = document.body.scrollHeight;
    parent.postMessage(height,url);
    })
    if(pageurl.split(langss + "/")[1]) parent.postMessage(pageurl.split(langss + "/")[1],url);
    

    if(document.querySelector("#notFound")) parent.postMessage("我404了",url);

</script>
</body>
</html>
